Repository navigation
feat(openai): let a host turn native tool calls on for a local runtime - #154
Conversation
build_local_runtime_chat_model always set native_tool_calling = false. That is the right default for Ollama or LM Studio, but local-openai also fronts hosted OpenAI-compatible gateways whose models call tools natively and whose servers parse <tool_call> text in the model's own template. Prompted with the JSON-in-tag protocol there, the gateway reads the JSON as the tool's name. Add build_local_runtime_chat_model_with_native_tools, which takes the choice from the host and keeps the rest of the local-runtime profile. The old builder delegates to it with false, so existing callers are unchanged. Co-authored-by: Medulla <medulla@tinyhumans.ai>
Tiny Sweeper reviewReviewed head: Verdict: Ready for maintainer review · Priority: none Diagnostics and supporting detailsFull summaryPull request purpose (title): feat(openai): let a host turn native tool calls on for a local runtime. This pull request adds a host-controllable opt-in for native tool calling on the `local-openai` runtime, which previously hard-coded `native_tool_calling: Some(false)`. The motivation in the title and description is that a model served through an OpenAI-compatible gateway (glm-5.3-flash on Surplus) fails under the JSON-in-tag text protocol with `unknown tool` errors and abandoned builder jobs, while native tool calls work there. The change introduces a builder variant that takes the host's choice, keeps the existing builder's always-off behaviour by delegating with `false`, and re-exports the new function. The latest-push delta is unavailable, so nothing here describes a push-level delta. Review snapshot
Completeness: Complete What changedA new public builder `build_local_runtime_chat_model_with_native_tools` in `crates/tinyinference-llm/src/providers/openai/config.rs#pub fn build_local_runtime_chat_model(` area threads a host-supplied `native_tool_calling` flag into the config instead of the fixed `Some(false)`, leaving the rest of the local-runtime profile (single system message, no vision, `num_ctx`, stream-backed invoke) intact. The existing `build_local_runtime_chat_model` now delegates to it with `false`, so current callers are unchanged, and the new function is re-exported from `crates/tinyinference-llm/src/providers/openai/mod.rs`. A test in `crates/tinyinference-llm/src/providers/openai/config_tests.rs` exercises both the opt-in and default branches. Latest-push delta: unavailable — the review state has no trusted base-to-head diff comparison, so no push-delta content is reported. Features
TestsNo supported feature-to-test mapping was produced. Test execution is not inferred. FindingsNo active actionable findings. Before mergeNone. Agent review detailscritique
security
tests
commits
description
Evidence and run details
|
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
…ols) Picks up build_local_runtime_chat_model_with_native_tools (tinyhumansai/tinyinference#154) so OpenHuman can turn native tool calls on for a local-openai endpoint that supports them. Co-authored-by: Medulla <medulla@tinyhumans.ai>
Why
Teeny's builder agent runs
glm-5.3-flashon Surplus through OpenHuman'slocal-openairuntime.build_local_runtime_chat_modelhard-codesnative_tool_calling: Some(false), so the harness sends notoolsand prompts the model with the JSON-in-tag text protocol instead. On Surplus that breaks. After the first call, every builder call failed withunknown tool \` (arguments: {})`, and every builder job ended "abandoned: the builder never reported back".Native tool calling works for this model on Surplus. A host has to be able to say so.
Probes against Surplus (glm-5.3-flash, OpenAI-compatible), run on the Teeny box:
tools, model asked to emit JSON-in-tag: notool_calls; the text stays text.tools=[shell], a normal ask: a clean native call,finish_reason=tool_calls, idcall_7df8…, nameshell, args{"command":"ls"}. Native tool calling works for this model on Surplus.toolssent plus "don't use native calls, write JSON-in-tag", non-stream: content came back verbatim,tool_callsnull.stream=true: only thinking chunks and no answer. Surplus attachesresponse_error: {format_error, content_validation: ["valid_think_chunk"], tool_calls_validation: [], extra_tags: ["only_thinking"]}. Surplus validates and rewrites output server-side, so the text protocol plus thinking is fragile there.What
build_local_runtime_chat_model_with_native_tools(..., native_tool_calling: bool), re-exported beside the existing builder. It is the same local-runtime profile (single system message, no vision,num_ctx, stream-backed invoke forlocal-openai) with native tool calling chosen by the host.build_local_runtime_chat_modeldelegates to it withfalse, so every existing caller is unchanged.Test
providers::openai::config::tests::local_runtime_builder_can_opt_into_native_toolserror[E0425]: cannot find function build_local_runtime_chat_model_with_native_toolscargo test -p tinyinference-llmpasses (882 lib tests plus the integration suites), andcargo clippy -p tinyinference-llm --all-targetsis clean.Consumers: tinyagents bumps this crate; OpenHuman then reads an opt-in
local_ai.native_tool_callingsetting and passes it here.Summary by CodeRabbit